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This  grant  supported  a  range  of  theoretical  work  on  topics  in  neural  networks,  learning  theory, 
dynamical  systems  and  optimization.  We  summarize  the  achievements  below  and  give  a  list  of  the 
relevant  publications. 

(a)  LIST  OF  ACHIEVEMENTS. 

It  is  convenient  to  divide  the  work  into  four  parts:  (i)  dynamical  systems  models  of  compu¬ 
tation,  (ii)  learning  and  reasoning  theory,  (iii)  clocked  neural  networks  and  quantum  computing, 
and  (iv)  statistical  physics  models  of  neural  networks. 

1.  Dynamical  System  Models  of  Computation, 

We  found  a  formalism  for  describing  the  dynamics  of  hybrid  systems  and  studied  their  caba- 
bilities  as  computers.  Our  models  allow  us  to  look  at  certain  bctsic  processes  such  as  analog  to 
digital  conversion,  in  an  abstract  way  emphasizing  the  fact  that  digital  representations  need  to  be 
robust  whereas  analog  ones  are  continuously  variable.  Through  the  study  of  these  models  we  are 
able  to  see  certain  problems  in  computer  science  and  signal  processing  in  a  common  framework. 
We  have  found  certain  classes  of  differential  equations  with  inputs  whose  responses  have  readily 
identifible  characteristics  which  depend  only  on  topological  aspects  of  the  inputs  and  not  on  the 
detailed  behavior  of  the  input.  These  systems  a  can  be  thought  of  as  defining  automata  and  are 
capable  of  performing  certain  computations  in  an  extremely  robust  way.  See  references  [1-7]. 

2.  Learning  and  Reasoning  Theory. 

We  proved  that  many  models  of  reasoning,  and  approximate  reasoning,  commonly  used  in 
Artificial  Intelligence  are  computational  infeasible,  in  the  sense  of  being  NP-complete.  We  put 
complexity  bounds  on  approximate  reasoning  for  classes  of  propositional  formulae.  Learning  theory 
was  applied  to  learning  visual  concepts.  A  book,  “Circuits  of  the  Mind",  was  published  describing 
a  theory  of  the  brain  based  on  neuron-like  elements,  called  neuroids  (see  reference  [21]).  This 
book  emphasized  the  importance  of  considering  the  computational  feasibility  of  algorithms  and  of 
exploiting  the  high  degree  of  connectedness  in  the  brain.  The  book  included  models  for  cognitive 
abilities  such  as  learning  (with  or  without  a  teacher),  memorization,  and  reasoning.  In  addition 
it  outlined  the  development  of  a  theory  for  learning  to  reason.  See  references  [8],  [10-14],  [16-18], 
[20-21]. 

3.  Clocked  neural  networks  and  quantum  computing. 

We  completed  an  analysis  of  a  new  clocked  neural  network  architecture  -  competitive  neural 
networks  -  optimized  for  visual  pattern  recognition.  Small  groups  of  fully  interconnected  elec¬ 
tronic  elements  are  used  to  recognize  elemental  pattern  features.  The  state  of  each  small  group, 
i.t.  the  most  favored  features,  are  then  compared  over  larger  distances,  using  relatively  few  long 
range  interconnections,  in  order  to  stitch  together  larger  objects.  In  a  series  of  two  long  papers 
(references  [23-24])  we  presented  a  detailed  analysis  of  the  gloabl  stability  of  competitive  networks, 
and  analyze  the  performance  of  associative  memory  using  this  architecture.  We  obtained  “phase 
diagrams”  with  regions  of  guaranteed  stability,  which  can  be  used  for  system  design.  These  ana¬ 
lytical  results  were  in  good  agreement  with  numerical  stability.  We  constructed  double  and  triple 
quantum  dots  with  tunable  coupling,  using  electron  beam  patterned  gates  on  top  of  GaAs/AlGaAs 
heterostructures.  Such  devices  can  be  regarded  as  artificial  molecules  with  tuneable  molecular  cou¬ 
pling.  The  Coulomb  blockage  is  used  in  low  temperature  measurments  to  determine  the  existence 
and  energetics  of  melecular  electronic  states,  whicha  re  superpositions  of  the  electron  states  on  the 
separate  quantum  dot  sites.  A  strong  splitting  of  Coulomb  blockage  conductance  peaks  into  two 
(and  three)  peaks  as  the  interdot  tunneling  is  increased  for  double  (and  triple)  dots  confirm  the 
existance  of  coherent  molecular  states  [25].  This  project  is  being  transferred  to  other  sources  of 
funding.  We  plan  measuremnets  of  the  molecular  states  for  other  interconnection  topologies,  and 
plan  to  test  the  sensitivity  of  coherent  states  to  electromagnetic  radiation.  See  references  [22-25]. 

4.  Statistical  Physics  models  of  neural  networks. 

We  used  mean  field  analysis  to  understand  the  properties  of  texture  models  near  their  phase 
transitions  temperatures.  We  developed  a  new  appraoch  to  self-organization  of  early  vision  and 
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showed  a  relation  to  the  influential  work  of  Becker  and  Hinton.  We  showed  that  algorithms  based 
on  statistical  physics  could  be  applied  to  calculate  robust  principal  components.  We  analyzed  de¬ 
terministic  annealing,  related  it  to  other  optimization  algorithms,  and  proved  convergence  bounds 
for  the  linear  assignment  problem.  We  obtained  convergence  bounds  for  learning  using  Radial  Ba¬ 
sis  functions  and  related  networks.  We  showed  that  realistic  neuronal  models  could  act  as  content 
addressable  memories.  It  was  dmonstrated  that  statistical  physics  algorithms  could  be  used  to 
perform  robust  principal  component  analysis.  See  references  191,  [15],  [19]  [26-32] 
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